Adaptive Collaborating Filtering

In this paper, we study collaborative filters that adapt future recommendations based on feedback (like/dislike) received for past recommendations. We consider a mathematical model where users and items are clustered, and ratings are noisy. We apply sequential decision techniques to study structure of collaborative filters that maximize the time average of the expected ratings. While our previous work considered the case with unknown clusters and a moving horizon reward, in this paper we study the case of known clusters and the long run average reward (that is, the ergodic cost). We show that our problem fits the Markov decision process framework. We show that if a solution to the dynamic programming equation exists, then the optimal cost is achieved for the problem. Moreover, we identify a simple collaborative filter that is optimal in the low noise regime. Moreover, empirically we find that this collaborative filter is near-optimal over the entire range of noise values.